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DUAL BUFFER VIDEO DISPLAY SYSTEM FOR 
THE DISPLAY OF ASYNCHRONOUS IRREGULAR 
FRAME RATE VIDEO DATA 
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BACKGROUND OF THE INVENTION 
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In computer-based video commTtrriration systems, a 
video signal is obtained from the camera at a constant 
frame rate bet, after transmission across die asynchro- 40 
nous or non-ideal network, the frames arrive at irregu- 
lar intervals. Some frames arrive early, some are de- 
layed, and bunching can occur. The display device at 
the receiving terminal, however, generally requires a 
constant frame rate supplied to it (e^» to match the 45 
raster scan rate of a CRT). In such systems it is there- 
fore necessary to match the irregular arrival of frames 
over the network with the constant supply required to 
the output screen. 
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. The designer of 
computer based video commnnirafion systems is there- 60 
fore faced with the problem of how to achieve regular 
play-out of the asynchronous incoming video signal 
while, at the same time, minimising the number of buff- 
ered video frames. 
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- nominal arrival time can be properly displayed. Only if 
a frame arrives more than T(L) late, will the AVK and 
areolar buffer empty and the video image will freeze. 
To decrease the risk of buffer starvation, the butler size 

5 can be increased to make T(L) Larger, but with a 15 
frames per second transmission rate, storing only 10 
frames adds a delay of } second. If the effectiveness of 
interactive applications such as video conferencing is 
not to be seriously degraded, only a handful of frames 

10 can be buffered with T(L) correspondingly small. 

The control process is responsible first for receiving 
data into the circular buffer, and then for forwarding it 
to the AVK. There is no control over output from the 
AVK, which is at a fixed rate. As explained in more 

15 detail below, the AVK requests frames from the circu- 

hi video conferencing or other inter- 
active applications where the overall amount of buffer- 
20 tog is limited, there may occasionally be particularly 
long delays on the network during which time the cir- 
cular buffer empties. In this case, the control process 
reacts by loading the AVK with null frames. A null 
frame b essentially the same as the preceding frame, so 
25 that, as far as the viewer is cencerned, video image 
temporarily freezes. Thus, each time the control process 
fails to find frames in the circular buffer, the requisite 
number of null frames are loaded into the AVK instead. 
Although the user may not notice the insertion of 
30 individual null frames, each null frame adds to the over- 
all delay in the system (Lc, it is effectively another form 
of buffering). If more and more null frames are inserted 
into the video stream, then this will, again, lead to an 
intrusive delay between transmission and display. This 
With reference now to FIG. 2, the mcoming video 35 problem can be overcome.by the circular buffer throw- 
signal, from the communication subsystem 15 arrives at ing away real data when the delayed frames do finally 
the workstation 13 for display on the associated monitor arrive. These frames are then effectively lost allowing 
9. The signal is transferred first to a buffer 23, and then the displayed image to catch up with the incoming 
to the AMU card 125 or, mare 'particularly; to the Au- signal It is the presence of two buffers that gives the 
dioVisual Kernel (AVK) mterface buffer 25 of the 40 flexibility to lose frames in this way, and so cope with 
AMII card. The buffer 23 provides a FIFO queue, occasional delays longer than T(L). 
conveniently implemented as a circular buffer. A con- The technique used to discard frames exploits the fact 
trol process 27 is responsible first for reading mcoming that, due to the Kmftwrf bandwidth of the channel the 
data into the circular buffer, and then for transferring video signal is compressed before transmission over a 
data from the circular buffer to the AVK. 45 computer based communication line, basically, two 

types of compression, spatial and temporal, are used In 
the former, the redundancy within a single frame is 

..■ removed, for example, by using the feet ihat adjacent 

pixels often have closely related brightness and color 
However, the trans- 50 values. A frame encoded using only spatial compression 
mission rate over the network is variable, depending on is known as a "still frame". Temporal compression 
network load, etc, so that the arrival, rate at the end of achieves a further level of cornpression by exploring 
the computer subsystem departs from this 15 Hz clock. the fact that the luminosity and color of the samVpixel 
Changes in CPU activity at the source and destination in two consecutive frames are, again, likely to be hi 
computers can also lead to variations in the effective 55 correlated. Therefore, in temporal compression, a frame 
frame arrival rate. Individual frames can have either a is encoded as a "relative frame" in terms of its ctiffer- 
posrtive or negative offset from their nominal arrival ence from the previous frame (we assume that a relative 
time, although it is assumed that frames do, in tact, frame is also spatially compressed). The greatest reduc- 
arrive in the correct sequence. It should be noted that ties in data is achieved if every frame (apart from the 
the variation in arrival times is such that even if the 60 first) is a relative frame, but this is highly error prone 
hardware could display each frame directly on arrival, since the loss.of a single frame will produce defects that 
the resulting sequence would be so temrxjrally distorted persist for all subsequent frames. Therefore, as a com- 
as to be unwatchable. promise, every Nth frame can be sent as a still frame, 
essential with all intervening frames as relative frames, so that 
Together, the AVK and circular buffer compensate 65 the result of compression is a regularly spaced series of 
for the variable arrival rate of the video frames by intro- frames whose size varies somewhat according to the 
ducing a time-lag, T(L), between the received and dis- temporal and spatial content of the data and, of course, 
played images. Any frame arriving within T(L) of its whether that particular frame is a still or relative frame. 
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When tr» buffer is not full, then incoming frames 
can be added to the buffer in the normal way. However, 
when the buffer is fall, there are two possible actions. If 10 
the incoming frame is a still frame, then the entire buffer 
is flushed before the incoming still frame is added to the 
queue. Alternatively, if the inrammg frame is a relative 
frame, then only relative frames below (Le^ that arrived 
earlier than) a still frame are flushed. This is because the 12 
previous soil frame is still required to make sense of the 
relative frames. In either case, flushing the buffer results 
in some frames being thrown away , and so the displayed 
image catch^up shghtly with the received image. 



25 



Turning now to the AVK, frames arc read out from 
the AVE for display at a filed rate. This leads to the 
possibility of buffer starvation if the AVK contains no 
more frames to read out to the screen. In such an even- 35 
tuality, the AVK pipeline needs to be reset, requiring a 
considerable system overhead during which time the 
video image is not updated, in contrast to the circular 
buffer, which can be emptied and refilled without pen- 
alty. « 

Accordingly, a lower limit, V(L), is set for the num- 
ber of frames in the AVK. This value is selected to 
substantially preclude buffer starvation yet, at the same 
time, not introduce an unacceptable delay. The control 
process responsible for transferring frames from the 45 
circular buffer to the AVK. then tries to maintain the 
number of frames in the AVK as close as possible to but 
sliehtlv above V(Ll 



55 



Once the control process has determined the number 
60 of frames to transfer to the AVK, it can either send this 
as a single request, or as an appropriate number of re- 
quests for individual frames. In the latter case, the circu- 
lar buffer can respond simply to each request by trans- 
ferring a frame if available, or inserting a null frame if 
65 not 
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